Search and retrieval of two-dimensional (2D) images in a database of 2D images using a sample image and based on similarity between the images is well known in the field. The similarity can be optionally based on different aspects. For example, a sample object shown in the sample image is searched in the database, and the images showing objects similar to the sample object are retrieved as search result. Retrieval methods usually calculate features for the images in the database and compare these features with a sample image or sample data to find similar images. The features can include, for example, color histograms, edges and feature points. The performance of the search generally depends on the quality and robustness of the feature analysis.
There are existing issues and problems for calculation of robust features. First, it is difficult to calculate and extract reliable features, such as edges or feature points, from blurred images. For images including transparent or partially transparent object, color histograms, edge detection and feature point detection would suffer from the background that is visible behind the transparent or semi-transparent object. Scattered objects such as a tree with branched and leaves would expose a large and spatially scattered range of depth. Moreover, for objects seen from different viewpoints among multiple images, geometrical and photometric distortions would make the image retrieval difficult.
Four-dimensional (4D) lightfield of a scene has been introduced as an alternative choice for image capture. A 4D lightfield can be captured, for example, by a plenoptic camera or a grid of cameras and allows refocusing of the capture after the snapshot, estimating the depth of the scene in the capture and many other flexible editing options for the capture data. Multiple classical 2D images can be calculated from a 4D lightfield data for a same scene. The calculated 2D images, so called lightfield images, may differ in viewpoint, viewing direction, resolution, depth of view, focal plane, etc. For example, a method of calculating refocused 2D images from a lightfield is illustrated in [I], where sub-images from the raw image of the camera sensor are spatially shifted in the image plane and subsequently summarized.
It can be expected that more lightfield data will be available and used in the near future. The techniques and solutions for search and retrieval of the lightfield data will also become essential for users.